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Introduction 


Mathematical  programming  problems  have  the  general  form  of  maxi¬ 
mizing  a  function  g(x)  with  respect  to  a  vector  x  which  is  restricted 
to  be  in  some  feasible  region  R.  Let 

* 

G  ■  max  g(x) • 
xeR 

Unfortunately,  unless  g  and  R  are  extremely  well-behaved  it  is  quite 

* 

often  impractical  to  find  an  xeR  with  g(x)  =  G  .  Hence  the  literature 
abounds  with  heuristic  procedures  purporting  to  provide  "good"  or  "near- 
optimal"  solutions  for  special  cases  of  g  and  R.  A  difficulty  with 
using  these  procedures  in  specific  problems  is  that  it  is  hard  to  de¬ 
termine  how  close  such  an  approximate  solution,  say  x,  is  to  being 

*  A 

optimal;  in  particular,  how  small  is  G  -  g(x) . 

The  objective  of  this  paper  is  to  indicate  some  relatively  new 

* 

statistical  procedures  for  obtaining  an  upper  confidence  limit  on  G  . 
Each  of  these  procedures  results  in  a  statement  that  with  a  chosen  con¬ 
fidence  (say,  a  95%  confidence)  G*  does  not  exceed  the  computed 
-* 

confidence  limit,  G  ;  that  is 

P(G*  ^  G*)  *  .95  . 

* 

The  procedures  do  not  Indicate  a  feasible  x  with  g(x)  *  G  or  even 
necessarily  produce  a  feasible  x  with  g(x)  near  G  .  Therefore,  a 
confidence  limit  procedure  is  not  Intended  as  a  replacement  for  a 
"solution"  finding  algorithm,  but  rather  it  is  intended  as  a  supplement 
to  such  an  algorithm  in  the  sense  that  the  difference  G  -  g(x)  pro¬ 
vides  a  needed  indication  of  just  how  "near"  the  "near-optimal" 
solution  is  to  being  optimal. 

The  following  situation  illustrates  one  type  of  setting  in  which  a 
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confidence  limit  procedure  can  be  used.  A  steepest  ascent  procedure  Is 
used  to  find  the  xeR  which  maximizes  g.  The  procedure  requires  a 
feasible  x  as  a  starting  point.  Hence  10  starting  points  x^,  ...,  x1Q 
are  selected  at  random  from  R.  Then  the  steepest  ascent  procedure  is 
carried  out  beginning  at  each  starting  point.  The  10  corresponding 


■optimal" 

solutions 

xr 

« « t )  h&d 

gl  = 

g(^1)  = 

3.7 

,  g2  «  g(x2)  * 

7.5 

g3  = 

3.9 

A 

,  g4  =  g(xA)  - 

3.6 

85  = 

g(x5)  = 

9.8 

,  g6  -  g(x6)  - 

4.3 

g7  * 

g(x?)  = 

1.1 

,  gg  =  g(x8)  - 

7.6 

g9  " 

g<x9)  “ 

7.5 

’  810=  8(x10)= 

4.1 

Let  the  ordered  values  of  this  independent  sample  of  g  values  be  denoted 
by  g(1),  g(2),  g(1Q)  where  g(1)  £  «(2)  £  •••  £8(10)‘  71,60  the  con~ 

fidence  limit  procedure  would  use  the  sample  size,  n  *  10  here,  and  the 

ordered  values  g^,  8(n)  t0  Produce  an  approximate  95%  upper  con- 

**  *  * 
fidence  limit  G  on  the  unknown  G  .  The  G  indicates  roughly  where  G 

* 

is  and  the  difference  G  -  8^q)  indicates  how  near  is  to  G  . 

The  chosen  confidence  level,  say  95%,  means  that  approximately  95%  of  the 
time  when  an  independent  sample  of  g  values  g^,  ...,  g^  are  observed  and 
G  computed 

8(n)  £  G  -  G  ’ 

Integer  programming  problems  are  another  setting  in  which  a  con¬ 
fidence  limit  procedure  can  be  used.  In  these  problems  the  solution 
strategy  is  usually  to  Implicitly  enumerate  the  feasible  integer  solutions 
using  a  branch-and-bound  scheme.  In  such  a  scheme  the  important  task  is 
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to  determine  whether  for  say  fixed  values  of  x^,  there  exist 


feasible  x 


P+l’ 


,  x  such  that  the  objective  function  g(x. ,  . ..,  x  ) 


m 


m 


exceeds  the  best  value  of  g  found  so  far.  If  no  such  completion  x 

x  exists,  then  all  solutions  with  these  fixed  values  of 

x^,  ....  Xp  have  been  effectively  enumerated.  An  approximate  way  of 

determining  whether  any  such  completion  x  - ,  ...,  x  of  a  partial 

p+J.  in 

solution  x^,  ...,  Xp  exists  is  to  randomly  select  n  feasible  completions, 

calculate  g  for  each  completion,  and  determine  a  95%  upper  confidence 
A*  * 

limit  G  on  the  maximum  value  G  of  g  with  the  given  fixed  values  of 

x^,  ...,  x^  If  G  doesn't  exceed  the  best  value  of  g  found  so  far,  then 

all  solutions  with  these  fixed  values  of  x, ,  ....  x  are  considered  to 

1  P 

have  been  enumerated.  Thus,  since  a  relatively  small  sample  of  com¬ 
pletions  can  be  evaluated  much  faster  than  an  extensive  enumeration  of 
the  completions,  a  confidence  limit  procedure  can  greatly  facilitate  the 
solution  of  Integer  programming  problems. 

2.  Confidence  Limit  Procedures 

Three  confidence  limit  procedures  are  described  in  the  next  three 
subsections.  The  author's  computational  experience  with  these  procedures 
is  described  in  section  3. 

All  three  procedures  determine  the  confidence  limit  G  on  G  in 
basically  the  same  way. 

(1)  Begin  with  n  independent  sample  values  of  g  denoted  by 
gp  ...»  gn>  Denote  the  ordered  values  of  g^,  ....  g^ 


by  g, 


**  wlth  -  *•*  1  8<Vl  -  G 


>(1)’  R(n)  s(l)  -  B(2)  -  *  -  ®(n) 

(2)  Assume  that  the  probability  P(g)  of  a  sample  value  of  g 

* 

being  less  than  or  equal  to  say  g,  (g^  <_  g  <_  G  ),  is 
well  approximated  by  an  m-th  degree  polynomial,  say 


I 

£ 
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m 


F(g  ;a, 


* 

G  ) 


l  «J 
j=l  J 


<9 


*  1 
-G  y 


+  1  where  m  is  specified  but 


a, ,  . . . ,  a  are  unknown . 
i  in 

* 

(3)  For  a  given  estimate  G  of  G  calculate  estimates 

a, .  • « •  ■  oi  of  ot-  >  •  •  • ,  ct  • 
l  ml  m 

(4)  If  the  estimated  distribution  function  F(*;a,G)  is  a 

"good  fit"  to  the  observed  g. ,  g  ,  increase  the 

in 

estimate  G  of  G  and  repeat  (3) . 

A  ^ 

(5)  The  upper  confidence  limit  G  on  G  is  the  largest  value 
of  G  for  which  a  good  fit  is  obtained. 

The  three  procedures  differ  in  the  way  the  a^,  . a  are  de¬ 
termined  and  the  criterion  used  to  evaluate  the  goodness  of  the  fit  of 
F( •  ;oi»G)  to  gl,  ....  gn  . 

The  execution  of  the  confidence  limit  procedures  does  not  depend  upon 
the  specific  way  in  which  g^,  . ...  g^  are  obtained. 

•A  iff  ■  iff  A  A 

The  magnitude  of  |G  -  G  |  and  hence  the  usefulness  of  G  depends 

upon 

(1)  the  confidence  level  chosen  (95%  in  the  discussion  below)  ; 

(2)  the  shape  of  P( g  )  and  hence  the  way  in  which  the  sample 

values  g. ,  ...,  g  are  obtained;  and 
l  n 

(3)  how  close  P( g  )  is  to  being  an  m-th  order  polynomial. 


2.1  A  Confidence  Limit  Procedure  based  on  Sample  Proportions 

Before  the  sample  g^,  . ..,  is  observed,  choose  values  G^,  . . . ,  G^ 
such  that 

* 

G^  <  G2  ^  ....  ^  G^  ^  G 

and 

0  <  P(G1). 
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np  =  #  of  g  ,  g  <  G  ,  i  =  1,  ....  k. 

1  J.  n  —  l 

Then  p^,  p^  are  sample  proportions  having  a  multinomial  distribution 

with  (symmetric)  covariance  matrix  =  (v^^) 

where 

v^  =  P(Gi)  [l-P(Gj ) ]/n  for  i  <_  j. 

If  V11  =  (Vj^^)  »  then  a  goodness-of-f it  criterion  for  the  P^'s 
and  their  expectations  (the  P(G^)’s)  is  the  quadratic  form 


k  k 


Qik  -  £  £  vu/  [»1  -p<Gi>  Hpj-pcgj  : 


which  has  approximately  a  chi-squared  distribution  with  k  degrees  of 
freedom  (see  e.g.  Kendall  and  Stuart  (1960)  and  Watson  (1959)). 

A  A 

Furthermore,  if  =»  (v^)  where 


A 

vlij  =  for  1  1  1 


-_1  "(-1) 

and  if  *  (vj^  )  ,  then 


«ik-  l  l  [p^KCjSa.o'jHp -Kg  s«,g*>] 

i=l  j-1  J  j  j 

has  approximately  a  chi-squared  distribution  with  k  degrees  of  freedom. 

* 

For  a  given  estimate  G  of  G  determine  a  by  minimizing 


k  k 


Qlk  *  A  A  Vlij  Cpi“F<rTi5a,G)]tpj“F(Gj;a,G)] 


subject  to  the  linear  restrictions 
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dF(g';a,G) 

dg' 


g’  =  g« 


l  j«.  (g*  -  G)^1  >  0, 
j=l  3  S 


S  *"  1  |  •  «  •  )  S  y 


(2.1) 


(since  F( •;“»£)  should  be  a  distribution  function)  where  g|,  . .., 
are  a  relative  dense  set  of  grid  points  with  g^<g^<. . .  .<gg<G  . 

Having  solved  this  quadratic  programming  problem  using  any  of 

several  available  algorithms  (e.g.  Wolfe  (1959)),  accept  F(*;a,G) 

as  being  a  "good  fit"  to  g^,  g^  if  and  only  if  doesn't  exceed 

the  95-th  percentile  of  a  chi-squared  distribution  with  k  degrees  of 

A  * 

freedom.  Let  the  confidence  limit  G^  be  the  largest  G  for  which 
F(*;a,G)  is  accepted  as  being  a  "good  fit"  to  g^,  ...,  g^  . 

This  confidence  limit  procedure  was  first  reported  by  Hartley 
and  Pfaf f enberger  (1969) . 


2.2 


A  Confidence  Limit  Procedure  based  on  the  Largest  Order  Statistics 
* 

If  P(g)  =  F(g  ;a,G  )  ,  then  the  transformed  variables 

Zi  =  F^8(i)’a,G  )  »  1  =  n-k+1>  •••»  n  ,  (2.2) 


would  have  the  distribution  of  the  k  largest  order  statistics  from  a 

uniform  distribution.  Hence,  z  z  would  have 

n-k+1  n 


Efz^]  ■  i  /  n+1  ,  i  =  n-k+1,  ...,  n  , 
and  (symmetric)  covariance  matrix  =  (v2y)  where 

v2ij  «=  i(n-j+l)/(n+l)2(n+2)  for  i  <_  j  . 

A  goodness-of-f it  criterion  for  the  z^'s  and  their  expectations  is  the 
quadratic  form 
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Q,k  -  I  I  (i^i/n+lKz  -j/n+1)  v2ii>  (2.3) 

i=n-k+l  j-n-k+1  3  3 

where  =  (v^"^)  wlth 

V2ii)  “  (n-k+2)  (n+1)  (n+2)/(n-k+l)  ,  i  =  n-k+1 

*  2(n+l)(n+2)  ,  i  =  n-k+2,  . ..,  n 

v2^}  -  -(n+1)  (n+2)  if  |  i-J  |  «  1  , 

v<  «  ■  0  otherwise  . 

Lurie  (1971)  and  Hartley  and  Pfaffenberger  (1972)  provided  tables  of 

the  relevant  percentiles  of  >  and  these  are  reproduced  in  Appendix  F. 

* 

For  a  given  estimate  G  of  G  let 

*i  “  F<*(i);“’G) 

where  a  is  determined  by  minimizing 


)2k  *  l  l  (z  -i/n+l)(z  -j/n+Dvj 

i-n-k+1  j*n-k+l  J 


(2.4) 


subject  to  the  linear  restrictions  (2.1)  with  8(n_k+^j  1.  8^  •••  £  8g  £  G. 

Having  solved  this  quadratic  programming  problem,  accept  F(  •;“>(;) 
as  being  a  "good  fit"  to  gp  . ..,  g^  if  and  only  if  the  observed  value  of 

A  A 

Q2k  doesn't  exceed  the  tabled  95-th  percentile  of  Q£k  given  in  Appendix  F. 

A  £  A 

Let  the  confidence  limit  be  the  largest  G  for  which  F(*;o,G)  is  accepted 
as  being  a  "good  fit"  to  g^  ....  gn« 

A 

Unfortunately,  the  distribution  of  Q  k  is  very  different  from  that  of 
Q_,  ,  In  fact,  for  n  ■  40,  if 


O'  ”  CL,  with  z  z  ■  1, 

x2k  x2k  n-k+1  n  * 
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and 

=  95-th  percentile  of  (?2k  from  Table  F.l, 

then 

Q2k  i  Qju  =  5.83  <  Q2k  -  15.52  for  k  *  5, 

Q2k  -  Q2k  =  13 *54  -  Q2k  =  25  for  k  “  10‘ 

and 

Q2k  1  Q2k  =  24.23  <  Qjk  =  32.7  for  k  *  15. 

Thus,  it  is  fruitless  to  compare  Q^k  with  the  percentiles  of  C^k  given 
in  Tables  F.l,  F.2,  and  F.3.  Although  the  distribution  of  Q ^  doesn't 
depend  upon  P(g),  the  distribution  of  probably  does.  Nevertheless, 
the  empirical  evidence  reported  in  section  3  suggests  that  Q^k  can  be 
fruitfully  compared  to  the  Monte  Carlo  generated  percentiles  of 
given  in  Appendix  F  (Tables  F.4,  F.5,  ....).  These  percentiles  were 
generated  by  simulating  z^  k+1*  •••»  zn  directly  from  the  uniform  dis- 

A  A 

tribution,  determining  a,  and  calculating  Q^k*  Each  tabled  percentile 
was  based  on  1500  repetitions. 

2.3  A  Confidence  Limit  Procedure  based  on  Derivatives 

Liau,  Hartley,  and  Sielken  (1973)  proposed  a  modification  to  the 

confidence  limit  procedure  described  in  section  2.2.  The  proposal  uses 

the  transformed  variables  z^,  ...,  z^  and  uses  as  its  goodness-of- 

fit  criterion  for  the  z^'s  and  their  expectations.  However,  for  a  given 
*  , 

estimate  G  of  G  the  z^  s  are  estimated  by 

zi  ”  «j(e(i)-c)J  +  1  +  a0 

where  the  an,  . ...  a  are  determined  by  doing  an  unrestricted  minimization 
u  m 

of 
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I  l  (*.-i/n+l)(z  - 

i=n-k+l  j-n-k+1  1  -1 


i/n+l)v' 


(2.5) 


The  difference  between  and  0  is  an  additional  indication  of  the 

* 

suitability  of  G  as  an  estimate  of  G  . 

Since  the  <s's  are  unrestricted  in  (2.5),  (2.5)  corresponds  to  a 
weighted  least  squares  fit  of  the  z^'s  to  their  expectations. 


Therefore, 


where 


'  T  -1  -1  T  -1 

a  -  (X*  V2X  XG)  XG  V2XY 


*  j[ (n-k+1) / (n+1^-1 ,  ....  fi/(n+lj-lJT 


1  (8(n-k+l)G)  *“  (8(n-k+l)_G) 


'g  : 


1  <8(n)~G)  •“  (8(n)~G) 


Since  the  a  in 

m 

F(g  ;a,G)  -  l  a,  (g  -G)J  +  1  +  a_ 
j-1  J 

was  determined  without  any  restriction  on  c»q  and  the  derivatives  of 
the  goodness  of  the  fit  of  F  relative  to  g^,  ...,  gR  is  evaluated  in 

A  » 

terms  of  and  the  derivative  of  F  between  g^  an<*  G.  In  particular. 


def ining 


8(n+i)  “  8(n)  +  m  (G~8(n)> '  1  "  1#  m  » 


X  «  a 
0  0  * 


A  m  —X  (a*,  a  G) 
i  do' 


Jul  5  V^n+i)"0^  ’ 


*(n+i) 


i  -  1 


*  •  •  •  f 


►■i  i 
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then  X  should  be  approximating  zero  and  the  X,,  ...»  X  should  be 
u  1  m 

approximating  nonnegative  derivatives.  Since 


where 


X  =  (Xq,  X1 . Xffl)  =  Ho 


1  0 


0  1  2(g(n+l)"G)  “^(n+l)"®5 


°  1  2(g(n^)"G) 


m(g(n+m)-G) 


X  is  approximately  a  multivariate  normal  vector  with  covariance  matrix 

E  =  H(X^V-^X  )  Thus,  for  some  X  =  (0,X, ,  ....  X  )T  with  X.  >  0, 

*  t  -1  * 

i  ■  1,  ...»  m,  the  =  (X-X)  E  (X-X)  should  have  approximately  a 

chi-squared  distribution  with  m+1  degrees  of  freedom  (see,  e.g. 

2 

Kshirsagar(1972))  .  Hence,  if  v  oc  is  the  95-th  percentile  of  a 

m+ l ) 

chi-squared  distribution  with  m+1  degrees  of  freedom,  G  is  accepted  if 
and  only  if 


-  _1  2 
(X-X)  I  (X-X)  <.  x 


X  —  (o ,  X ,  • « * »  X  )  >  0 

l  m  — 


m+1, .95. 


The  confidence  limit  is  the  largest  G  accepted. 


Of  course. 


(X-X)  i'1  (X-X) 


X  *  (0,1. ,  ...»  x)  ^  0 
x  m  — 


is  a  quadratic  programming  problem. 


3.  Observed  Empirical  Behavior  of  the  Confidence  Limits 

The  usefulness  of  Qlk  as  a  goodness-of-f it  statistic  for  a  com¬ 
pletely  specified  distribution  P(g)  is  well  understood.  In  addition 


Hartley  and  Pfaf f enberger  (1972)  and  Lurie  (1971)  have  shown  that  Q2k 
can  be  a  very  powerful  goodness-of-f it  statistic  for  a  completely 
specified  distribution  P(g) .  However,  the  distributions  of  the  goodness- 

^  a  a 

of-fit  statistics  Q^»  and  which  essentially  represent  the 

minlmums  over  the  parameter  spaces  of  Q  ,  respectively  are 

mostly  unknown.  The  original  hope  was  that  the  distribution  of  the  Q^’s 
would  be  sufficiently  close  to  that  of  the  Q^'s  to  make  the  Q^’s  useful 

in  determining  goodness-to-fit  for  not  completely  specified  distributions. 

*  * 

(The  F(*;a,G  )  are,  of  course,  not  completely  specified  since  o  and  G 

are  both  unknown.)  Unfortunately,  in  a  recent  Monte  Carlo  study  it  has 

become  apparent  that  this  hope  has  not  been  fully  realized.  The  relatively 

a. 

large  difference  between  the  distributions  of  Q2k  and  Q2k  has  already  been 
noted  and  at  least  partially  compensated  for  via  the  Monte  Carlo  sample 

A. 

percentiles  for  Q2k  tabled  in  Appendix  F. 

The  Monte  Carlo  study  was  itself  quite  reasonable  although  small. 

Each  of  the  four  distributions,  P(g),  illustrated  in  Figure  1  was  con¬ 
sidered.  In  each  case  the  range  of  the  distribution  is  from  0  to  10  with 
G  *  10.  Three  independent  samples  of  size  n  *=  40  were  selected  from 

*ft 

each  P(g)  and  95?  confidence  limits  G^,  G2,  G^  determined.  The  values 
k  »  5,  10,  20  were  considered  with  m  *  1  and  2  for  each  k.  When  de- 

'"if 

termlning  G^,  the  G^,  ...,  G^  were  2,  4,  6,  8,  10  for  k  =  5,  and  1,  2,  ..., 
10  for  k  ■  10,  and  .5,  1.0,  1.5,  ...,  10.0  for  k  «  20.  When  determining 

*  if  a  if 

G ^  and  G^,  the  number  of  grid  points  g^,  ...,  g^  at  which  the  derivative 
of  the  estimated  distribution  was  required  to  be  nonnegative  was  S  *  10, 
and  the  g|,  ...,  g  were  evenly  spaced  between  g^  and  G  for  G^  and 
evenly  spaced  between  8(n_k+1j  and  G  for  G2.  The  sequence  of  estimates  G 
of  G  considered  was  10.0,  10.1,  10.2,  10.3,  10.4,  10.5,  10.6,  10.8,  11.0, 


Figure  1.  Graphs  of  the  distribution  functions  P(g)  sampled  in  the 

Monte  Carlo  Study 


11.2,  11. A,  11.6,  11.8,  12.0,  12.5,  13.0,  13.5,  14.0,  14.5,  15.0,  15.5. 

* 

The  last  estimate  G  of  G  corresponding  to  a  "good  fit"  was  taken  to  be 
the  95%  confidence  limit  G  .  These  confidence  limits  are  given  in  Table  1. 

The  Monte  Carlo  study  indicated  several  interesting  characteristics 
of  the  confidence  limit  procedures. 

*  •k  A*  A* 

All  three  confidence  limits  ,  G 2>  G^  did  better  in  terms  of 

. A*  *i 

|G  -  G  I  for  m  =  1  than  they  for  m  =  2.  Some  additional  experimentation 
with  m  =  3  supports  the  idea  that  m  =  1  is  probably  best. 

Three  practical  limitations  to  the  confidence  limit  G^  are 

* 

(i)  the  necessity  of  specifying  apriori  G^  <  <  . . .  <  G^  <  G 

with  0  <  P(Gj)  <  P(G2)  <  ...  <  P(Gk)  <  1, 

* 

(ii)  the  singularity  of  if  P(G^)  =  0  or  >_  G  for  any  i,  and 

(iii)  the  frequent  singularity  of  if  k  is  large  relative  to  n  so 
that  two  or  more  sample  proportions  (p^)  are  equal. 

This  last  limitation  made  useless  most  of  the  time  when  k  =  10  or  20 
and  n  =  40. 

A*  A* 

The  confidence  limits  G2  and  G^  tended  to  improve  as  k  increased. 

Although  the  four  P(g)  graphs  in  Figure  1  are  certainly  not  all  linear, 

they  do  not  depart  "drastically"  from  linearity,  so  it  is  entirely  possible 
a*  *  * 

that  G2  and  G^  might  not  always  improve  with  k  for  other  shapes  of  P(g) . 

Unfortunately,  for  small  k  (k  =  5,  10)  the  distribution  of 

seems  to  not  be  close  enough  to  a  chi-squared  distribution  with  m  +  1 

'  * 

degrees  of  freedom  to  make  G^  very  useful.  From  a  theoretical  viewpoint 

*  T  -1  -IT  -1 

the  problem  is  in  the  "approximate"  normality  of  a  *  (X^V2  Xg^  ^GV2  ^ 
since  it  is  the  which  is  random  and  not  the  Y.  From  a  practical 
viewpoint  the  problem  is  that  almost  any  reasonable  a  has  0, 

i  ■  1,  ...»  m.  Hence  Q,.  becomes  almost  entirely  based  on  |an  -  01  . 


<n  > 
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Table  1.  Approximate  95%  confidence  limits  for  G  =  10  based  on  40 

observations  from  the  P(g)  distributions  in  graphs  1,  2,  3,  4* 


Graph  1 


Confidence  *  |  Graph  2  I  J  I  ^raPn  <♦ 

Limit  m  k  Trial:  123123  123123 


Graph  3 


Graph  4 


1  5 


10.210.110.1  10.010.010.3  10.110.010.1  10.010.110.1 


1  10 


10.2  10.1  10.2  10.0  —  — 


—  10.2 


1  20 


2  5 


13.5  13.5  13.5  11.0  —  14.5  10.2  10.0  10.1  13.0  13.5  13.0 


2  10 


—  13.5  13.5  10.3  —  — 


2  20 


1  5 


11.6  11.0  11.2 


15.0  — 


10.5  —  14.0 


10.0  10.0  10.1 


1  10 


13.0  11.6  11.2  11.2  11.6  14.5 


12.5  13.0  13.0 


10.0  10.0  10.3 


1  20 


13.0  11.2  11.4 


10.0  10.5  10.0 


10.0  11.0  11.0 


10.0  10.0  10.0 


2  5 


13.5  12.0  13.0 


10.0  — 


—  10.0  10.3 


2  10 


11.6  14.0 


15.0  — 


10.0  10.0  10.1 


2  20 


-  15.0  - 


10.0  —  15.0 


10.0  10.0  10.0 


1  5 


1  10 


1  20 


11.811.611.2  11.412.514.5  13.012.511.2 


—  —  14.5 


2  5 


2  10 


2  20 


—  14.5  14.5 


*A  —  Indicates  that  none  of  the  trial  estimates  of  G  yielded  a  goodness- 


of-fit  index  exceeding  its  critical  95-th  percentile. 
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A  * 

But  even  concentrating  on  <Jq  and  evaluating  an  estimate  G  of  G  on  the 

~  T  —1  —1  ^ 

hasis  of  the  "approximate"  normality  of  (a  -0)/[(X  V0  X  ).  .]2 

U  G  *L  G  1  9 1 

~  *  * 

doesn  t  yield  a  G  close  enough  to  G  often  enough  to  be  useful. 

*  k  *  '"k  A  k 

The  champion  among  G^,  G2,  and  G^  appears  to  be  G^.  The  confidence 

*  * 

limit  G^  probably  has  the  strongest  intuitive  appeal,  and  the  usefulness 
of  as  a  goodness-of-f it  index  for  a  completely  specified  P(g)  also 
makes  attractive. 

* 

As  one  might  expect,  the  more  nonlinear  P(g)  is  near  G  and  the  nearer 

1  * 


dP(g) 


is  to  zero  the  tougher  it  is  to  estimate  G  . 


In  fact,  the  ranking  for  the  slopes  of  P(g)  near  G  for  graphs  1,  2,  3,  and 
4  was  4,  1,  3,  2  with  4  being  the  largest  and  the  ranking  of  the  per¬ 
formance  of  G^  was  the  same  4,  1,  3,  2  with  G^  having  the  best  performance 
for  graph  4.  This  is  particularly  heartening  from  a  practical  viewpoint 
since  graph  4  corresponds  to  a  g(x)  with  local  optimum  for  x  e  R  -  a 

situation  which  seems  to  occur  frequently  in  practical  mathematical  pro¬ 


gramming  problems. 

A  k 

The  behavior  of  G  appears  to  be  best  when  the  P(g)  distribution  is 

*  * 

linear  with  positive  slope  for  g  <  G  in  a  neighborhood  of  G  .  A  way  to 
transform  the  underlying  distribution  P(g)  into  a  distribution  having 
these  characteristics  more  closely  is  to  perform  the  confidence  limit 
procedures  on 


where  u^,  ...,  u^  are  Independent  uniform  deviates  with  say 

P(u  t)  *  0  ,  t  <  0 

1  * 

»  ~  ,  0  <  t  <  U 

U 

*  1  t  ^  l* 

* 

and  U  a  suitably  chosen  constant.  Of  course,  the  least  upper  bound  on 


the  g's  is  still  G  .  At  first  glance  it  seems  counter-productive  to 
introduce  noise  (the  u's)  into  the  observations  analyzed.  However,  when 
P(g)  is  nonlinear  with  slope  near  0  at  G  ,  the  additional  noise  in  the  g's 

f\j 

may  be  more  than  offset  by  the  relative  linearity  of  P(g) .  To  illustrate 
these  ideas,  the  above  Monte  Carlo  study  was  essentially  repeated  with 

*  O 

the  g^'s  transformed  to  g's  using  u's  that  were  uniformly  distributed 
on  [0,  U  ]  with  U*  =  .5,  1.0,  2.0,  and  5.0.  The  results  are  indicated 
in  Tables  2,  3,  4  and  5  respectively.  (Since  G^  appeared  to  do  better 
for  k  large  than  for  k  small,  k  =  5  was  replaced  by  k  =  40  in  this 
study.) 

Several  interesting  observations  can  be  made  concerning  the  outcome 
of  the  Monte  Carlo  study  of  the  uniform  deviate  transformation.  Several 
of  the  tentative  conclusions  based  on  the  original  Monte  Carlo  study 
were  confirmed;  namely, 

(i)  m  =  1  tends  to  be  best, 

*  ft 

(ii)  G^  requires  too  much  apriori  information  and  usually  can't  be 
determined  for  k  large  relative  to  n  (k  =  10,  20  here) , 

(111)  G2  and  tend  to  improve  as  k  increase,  and 

^ft 

(iv)  G 2  tends  to  perform  the  best  overall. 

* 

As  expected,  the  best  value  for  U  depended  on  the  graph  of  P(g) . 

* 

Tables  6  and  7  indicate  the  relative  behavior  of  for  U  =0.0  (the 
original  Monte  Carlo  study),  0.5,  1.0,  2.0,  and  5.0.  In  graphs  1,  2,  3, 

ft 

and  4  the  relative  magnitudes  of  the  slope  of  P(g)  near  G  is  roughly  the 
same  as  the  relative  magnitudes  of  P(9.0  g  10.0).  Hence  the  following 
comparison  is  revealing 


1 
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Graph 

P(9.0  <  g  _<  10) 

Best 

* 

U 

4 

.15 

0 

1 

.10 

1  or 

2 

3 

.05 

1  or 

2 

2 

.01 

5 

• 

Thus,  it  appears  that  the  greater  the  probability  of  g  being  close  to 
*  * 

G  the  smaller  U  should  be.  A  conclusion  which  agrees  with  most 


people's  intuition. 
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Table  2.  Approximate  95%  confidence  limits  for  G  =  10  based  on  40 
transformed  observations  from  the  P(g)  distributions  in 
graphs  1,  2,  3,  and  4  with  the  transformation  being  the 
subtraction  of  a  uniform  deviate  with  range  [0,  .5]* 


Confidence 

Limit 


Graph  1 

Graph  2 

Graph  3 

Graph  4 

k  Trial: 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

5 

10.3 

10.2 

10.2 

— 

10.6 

10.0 

— 

10.2 

10.0 

10.0 

10. 

10 

— 

10.2 

— 

— 

— 

— 

— 

— 

— 

— 

10.1 

— 

20 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

5 

14.0 

13.5 

13.5 

— 

— 

15.0 

10.2 

— 

10.2 

13.0 

13.5 

10. 

10 

— 

13.5 

— 

— 

— 

— 

— 

— 

— 

— 

13.5 

— 

20 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

5 

11.4 

11.4 

11.2 

14.5 

— 

— 

11.4 

— 

14.0 

14.5 

15.0 

10. 

10 

12.5 

11.6 

11.0 

12.5 

14.5 

12.5 

12.5 

12.5 

12.5 

10.0 

10.0 

10. 

20 

13.0 

11.4 

11.2 

10.0 

10.8 

10.0 

11.2 

10.0 

10.8 

10.0 

10.0 

10. 

5 

13.0 

13.0 

12.5 

— 

— 

— 

13.0 

— 

— 

— 

— 

10.  i 

10 

— 

11.6 

11.0 

— 

— 

— 

— 

— 

— 

10.0 

10.0 

11. 

20 

— 

15.5 

15.5 

15.0 

— 

— 

10.5 

15.5 

14.0 

10.0 

10.0 

10.1 

10 

13.0 

— 

— 

— 

— 

— 

— 

— 

15.0 

— 

— 

14.1 

20 

11.8 

11.6 

11.2 

11.4 

11.8 

14.0 

12.5 

12.5 

11.6 

— 

13.5 

12.! 

40 

11.0 

10.8 

10.6 

10.0 

10.0 

11.0 

11.2 

11.6 

11.0 

11.4 

11.2 

11.: 

10 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

20 

— 

— 

15.5 

— 

— 

— 

— 

15.0 

15.5 

— 

— 

— 

40 

15.0 

14.0 

11.4 

10.8 

12.5 

10.3 

15.5 

— 

15.0 

10.0 

15.5 

15.! 

*A  —  indicates  that  none  of  the  trial  estimates  of  G  yielded  a  goodness- 
of-fit  index  exceeding  its  critical  95-th  percentile. 


Table  3.  Approximate  95%  confidence  limits  for  G  =  10  based  on  40 


transformed  observations  from  the  P(g)  distributions  in 

graphs  1,  2,  3,  and  4  with  the  transformation  being  the 

subtraction  of  a  uniform  deviate  with  range  [0,  1.0]* 

Graph  1  Graph  2  Graph  3  Graph  4 

m  k  Trial:  123  123  123  123 

1  5  10.3  10.2  10.2  —  —  10.8  10.3  —  10.2  10.2  10.0  10.0 

1  10  10.4  —  10.2  —  —  —  —  —  —  —  — 

1  20  —  —  —  —  —  —  —  —  —  —  — 

2  5  14.0  13.5  13.5  —  —  15.5  10.3  —  10.6  13.5  13.5  10.0 

2  10  14.5  —  13.5  —  —  —  —  —  —  —  —  — 

2  20  —  —  —  —  —  —  —  —  —  —  —  — 


G2*  1  5  11.0  11.4  10.8  14.0  —  —  11.6  —  13.5  14.0  15.0  10.5 

1  10  11.8  11.6  11.0  10.8  11.8  14.0  12.5  12.5  12.5  10.0  11.8  10.1 

1  20  12.5  11.2  11.0  10.2  10.6  11.4  11.0  10.0  10.5  10.0  12.5  10.0 

2  5  12.5  13.0  11.8  —  —  —  13.5  —  —  —  —  11.4 

2  10  15.5  14.0  13.5  —  —  —  —  —  —  10.0  15.5  11.2 

2  20  —  15.5  15.0  —  —  —  10.5  15.0  14.5  10.0  —  10.0 

63*  1  10  —  —  14.0  —  —  —  —  —  —  --  —  10.6 

1  20  11.8  11.4  11.0  10.2  11.2  12.5  13.0  13.0  11.6  14.5  12.0  11.4 

1  40  10.8  10.8  10.4  10.2  10.0  10.4  11.2  10.0  10.8  10.8  10.8  11.0 

2  10  —  —  —  —  —  —  —  —  —  —  —  13.0 

2  20  —  —  15.0  15.5  —  —  —  15.0  13.5  15.5  —  15.5 

2  40  15.0  12.5  11.0  11.4  13.0  12.5  15.5  —  15.0  14.5  14.5  12.0 

* 

*A  —  indicates  that  none  of  the  trial  estimates  of  G  yielded  a  goodness- 


of-fit  index  exceeding  its  critical  95-th  percentile. 


Confidence 

Limit 


-  * 


o  > 
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Table  4.  Approximate  95%  confidence  limits  for  G  =10  based  on  40 
transformed  observations  from  the  P(g)  distributions  in 
graphs  1,  2,  3,  and  4  with  the  transformation  being  the 
subtraction  of  a  uniform  deviate  with  range  [0,  2.0]* 


Confidence 

Limit 

-  * 

G1 


m 

1 

1 

1 

2 

2 

2 


Graph  1 

| 

Graph  2 

Graph  : 

3 

Graph  4 

k 

Trial:  1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

5 

10.3 

10.2 

10.3 

— 

— 

— 

10.2 

10.0 

10.0 

10.3 

10.0 

10. 

10 

— 

10.3 

— 

— 

— 

— 

— 

— 

10.2 

— 

10. 

20 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

5 

14.0 

13.5 

14.0 

— 

— 

— 

10.3 

10.0 

13.5 

13.5 

14. 0j 

10. 

10 

— 

14.0 

— 

— 

— 

— 

— 

— 

— 

14.0 

1 

13. 

20 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

5 

10.0 

14.0 

11.8 

—  : 

13.5 

15.0 

14.5 

14.5 

14.0 

14.0 

11. 

10 

10.2 

13.0 

10.5 

11.2 

11.0 

15.0 

1 

13.0 

10-8  | 

i 

12.5 

12.5 

10.0 

11. 

20 

10.0 

12.0 

10.4 

10.0 

10.0 

11.0 

12.0 

10.0 

10.8 

12.0 

11.2 

11. 

5 

10.0 

— 

14.0 

— 

— 

— 

— 

— 

— 

— 

15.0 

12. 

10 

10.2 

— 

13.5 

— 

— 

— 

— 

14.5 

— 

— 

10.0 

14. 

20 

10.0 

— 

14.5 

— 

— 

— 

— 

14.0 

15.0 

1 

— 

15. 

10 

— 

15.0 

13.5 

— 

12.5 

— 

— 

— 

15.0 

— 

14.1 

20 

13.5 

11.8 

10.4 

10.4 

10.0 

14.0 

12.5 

10.6 

11.2 

12.0 

13.5 

11.: 

40 

10.6 

11.0 

10.0 

10.0 

10.0 

11.2 

11.0 

10.0 

10.6 

10.6 

11.4 

io.: 

10 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

20 

— 

— 

14.5 

13.5 

15.0 

— 

— 

14.5 

14.5 

— 

— 

14.: 

40 

14.0 

14.5 

10.2 

10.0 

12.0 

10.0 

15.5 

14.0 

11.6 

14.5 

13.0 

13.: 

*A  —  indicates  that  none  of  the  trial  estimates  of  G  yielded  a  goodness- 
of-fit  index  exceeding  its  critical  95-th  percentile. 
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Table  5.  Approximate  95%  confidence  limits  for  G  »  10  based  on  AO 
transformed  observations  from  the  P(g)  distributions  in 
graphs  1,  2,  3,  and  A  with  the  transformation  being  the 
subtraction  of  a  uniform  deviate  with  range  [0,  5.0]* 


Confidence 

Limit 


A  * 

G3 


Graph  1 

Graph 

2 

Graph  3 

Graph  4 

m 

k  Trial: 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

i 

5 

— 

10.3 

10.3 

— 

— 

— 

10.5 

— 

— 

— 

— 

10. 

i 

10 

— 

— 

— 

— 

— 

— 

10.5 

— 

— 

— 

— 

10. 

i 

20 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

2 

5 

— 

15.0 

15.0 

— 

— 

— 

15.5 

— 

— 

— 

— 

14. 

2 

10 

— 

— 

— 

— 

— 

— 

15.5 

— 

— 

— 

— 

14. 

2 

20 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

— 

1 

5 

14. 0 

12.5 

11. A 

11.4 

13.5 

10.8 

14.5 

— 

10.0 

10.0 

13.5 

12. 

1 

10 

1A.0 

11.0 

12.5 

10.0 

10.0 

11.0 

13.5 

13.5 

10.5 

10.0 

11.6 

12. 

1 

20 

12.5 

11.0 

11. A 

10.0 

10.0 

10.0 

13.5 

11.4 

10.0 

10,0 

10.1 

11. 

2 

5 

— 

15.5 

13.5 

— 

— 

14.5 

— 

— 

11.8 

11.6 

— 

14. 

2 

10 

— 

15.5 

— 

15.5 

15.5 

— 

-- 

— 

14.5 

11.6 

— 

15. 

2 

20 

— 

— 

— 

14.0 

14.5 

15.0 

— 

— 

15.5 

13.0 

1 

10 

— 

13.0 

15.5 

11.4 

14.5 

— 

— 

14.0 

14.0 

— 

1 

20 

11.8 

10.6 

11.0 

10.0 

10.3 

10.0 

12.5 

13.0 

10.0 

10.0 

10.4 

11. 

1 

A0 

10.5 

10.0 

10.3 

10.0 

10.0 

10.0 

11.4 

11.6 

10.0 

10.0 

10.0 

11. 

2 

10 

— 

— 

— 

— 

— 

— 

— 

— 

“ 

— 

— 

— 

2 

20 

— 

15.5 

— 

13.0 

13.5 

14.5 

— 

15.5 

14.5 

13.5 

15.0 

— 

2 

A0 

— 

10.0 

10.6 

11.0 

12.5 

11.4 

— 

— 

10.0 

11.4 

13.5 

15. 

*A  —  Indicates  that  none  of  the  trial  estimates  of  G  yielded  a  goodness- 
of-fit  index  exceeding  its  critical  95-th  percentile. 
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“  * 


Table  6.  G ^  based  on  40  observations  from  the  P(g)  distributions  in 

*  * 
graphs  1,  2,  3,  4  when  G  =  10  and  a  uniform  0,  U 

deviates  are  subtracted  from  the  observed  g's* 


Graph  1 

Graph  2 

Graph  3 

Graph  4 

<rC 

U 

m 

k  Trial: 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

0.0 

1 

5 

11.6 

11.0 

11.2 

15.0 

— 

10.5 

— 

14.0 

10.0 

10.0 

10.1 

0.5 

1 

5 

11.4 

11.4 

11.2 

14.5 

— 

— 

11.4 

— 

14.0 

1.4 .5 

15.0 

10.3 

1.0 

1 

5 

11.0 

11.4 

10.8 

14.0 

— 

11.6 

— 

13.5 

14.0 

15.0 

10.5 

2.0 

1 

5 

10.0 

14.0 

11.8 

— 

13.5 

— 

15.0 

14.5 

14.5 

14.0 

14.0 

11.0 

5.0 

1 

5 

14.0 

12.5 

11.4 

11.4 

13.5 

10.8 

14.5 

— 

10.0 

10.0 

13.5 

12.5 

0.0 

1 

10 

13.0 

11.6 

11.2 

11.2 

11.6 

14.5 

12.5 

13.0 

13.0 

10.0 

10.0 

10.3 

0.5 

1 

10 

12.5 

11.6 

11.0 

12.5 

14.5 

12.5 

12.5 

12.5 

12.5 

10.0 

10.0 

10.2 

1.0 

1 

10 

11.8 

11.6 

11.0 

10.8 

11.8 

14.0 

12.5 

12.5 

12.5 

10.0 

11.8 

10.1 

2.0 

1 

10 

10.2 

13.0 

10.5 

11.2 

11.0 

15.0 

13.0 

10.8 

12.5 

12.5 

10.0 

11.6 

5.0 

1 

10 

14.0 

11.0 

. 

12.5 

10.0 

10.0 

11.0 

13.5 

13.5 

10.5 

10.0 

11.0 

12.0 

0.0 

1 

20 

13.0 

11.2 

11.4 

10.0 

10.5 

10.0 

10.0 

11.0 

11.0 

10.0 

10.0 

10.0 

0.5 

1 

20 

13.0 

11.4 

11.2 

10.0 

10.8 

10.0 

11.2 

10.0 

10.8 

10.0 

10.0 

10.0 

1.0 

1 

20 

12.5 

11.2 

11.0 

10.2 

10.6 

11.4 

11.0 

10.0 

10.5 

10.0 

!2.5 

10.0 

2.0 

1 

20 

10.0 

12.0 

10.4 

10.0 

10.0 

11.0 

12.0 

10.0 

10.8 

12.0 

11.2 

11.4 

5.0 

1 

20 

12.5 

11.0 

11.4 

10.0 

10.0 

10.0 

1 

13.5 

11.4 

10.1 

10.0 

10.1 

11.6 

*k  —  indicates  that  none  of  the  trial  estimates  of  G  yielded  a  goodness- 
of-fit  index  exceeding  its  critical  95-th  percentile. 
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*  * 

Table  7 .  Relative  behavior  of  G2  In  Monte  Carlo  study  of  the  uniform 
[0,  U  ]  deviate  transformation* 

“  * 

Average  G2 


* 


U  : 

0.0 

0.5 

1.0 

2.0 

5.0 

Graph 

k 

1 

5 

11.3 

11.3 

11.1 

11.9 

12.6 

1 

10 

11.9 

11.7 

11.5 

11.2 

12.5 

1 

20 

11.9 

11.9 

11.6 

10.8 

11.6 

2 

5 

— 

— 

— 

— 

11.9 

2 

10 

12.4 

13.2 

12.2 

12.4 

10.3 

2 

20 

10.2 

10.3 

10.7 

10.3 

10.0 

3 

5 

— 

— 

— 

14.7 

_ 

3 

10 

12.8 

12.5 

12.5 

12.1 

12.5 

3 

20 

10.7 

10.7 

10.5 

10.9 

11.7 

4 

5 

10.0 

13.3 

13.2 

13.0 

12.0 

4 

10 

10.1 

10.1 

10.6 

11.4 

11.0 

4 

20 

10.0 

10.0 

10.8 

11.5 

10.6 

*A  — 

indicates 

that 

in  at  least 

one  trial 

none  of 

the  trial 

estimates  G  of 

* 

G  - 

10  yielded  a 

goodness-of-f it 

index  ex- 

ceeding  its  critical  95-th  percentile. 
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Concluding  Remarks 

Certainly  the  final  word  on  a  practical  confidence  limit  G  has  not 
been  written.  In  this  paper  3  confidence  limit  procedures  based  upon  3 
goodness-of-f it  statistics  have  been  considered.  Of  these  the  procedure 
based  upon  C^,  m  *  1,  and  the  approximate  percentiles  of  0^^  has  had 
the  best  empirical  behavior. 

The  goodness-of-f it  index  given  in  (2.3)  can  be  rewritten  in 
the  form  s- 

Q2k  -  (n+1)  (n+2)  j  ^  +  *  d-*/  j  '  <"«> 

(See  Hartley  and  Pfaff enberger  (1972)).  For  m  =  1 

* 

—  F(g^^;a,G  ) 

=  1  +  a  (g(i)-G*) 

*  * 

and  the  a  which  yields  0^  for  a  given  estimate  G  of  G  is 

“  "  (G‘8(n-k+l))/[(n'k+1)T] 

with 

T  =  (G~8(n))  +  Jn_k+2(g(t)“8(t-l))  +  (G“8(n-k+l))  /(n-k+1) 

and 

Q2k  =  (n+1)  (n+2)|(n-k+l)_1  +  2a(g^_fc+1^-G)  (n-k+1)  1  +  a2  -  (n+2). 

*  "* 

Thus,  for  m  *  1,  and  G ^  are  easy  to  evaluate. 

The  confidence  limit  procedures  considered  herein  are  based  on  a 
goodness-of-f it  evaluation  of  F(*;a,G).  An  alternative  approach  taken  by 
several  authors  is  to  base  G  on  the  limiting  distribution  of  order 
statistics.  For  example,  Boender  et.  al.  (1980)  use  a  method  based  upon 
the  two  largest  order  statistics  and  a  result  due  to  De .  Haan  (1979). 
Robson  and  Whitlock  (1964)  also  base  their  confidence  limit  on  the  dis¬ 
tribution  of  the  two  largest  order  statistics,  but  they  assume  that  the 


J 


underlying  distribution  is  uniform.  Clough  (1969)  assumes  that  the 
g^'s  themselves  implicitly  act  like  largest  order  statistics  having  a 

limiting  distribution  of  the  exponential  form  and  determines  his  con- 

* 

fidence  limit  on  G  using  parameter  estimation  techniques  appropriate 
for  exponential  distributions.  Golden  and  Alt  (1979)  generalize 
Clough's  approach  by  assuming  that  the  limiting  distribution  is  of 
the  Weibull  form.  The  crucial  issue  concerning  these  alternative  ap¬ 
proaches  is  whether  or  not  in  a  particular  problem  the  sample  sizes 
(explicit  or  implicit)  are  sufficiently  large  for  the  limiting  dis¬ 
tributions  to  apply. 

The  "optimal"  transformation  g  of  g  has  not  been  identified  for 
confidence  limits  based  either  on  goodness-of-f it  criteria  or  on 
limiting  distributions.  The  usefulness  of  transformations  is  a  topic 
worthy  of  further  research. 

Although  neither  the  use  of  statistical  techniques  in  mathematical 
programming  nor  the  use  of  confidence  limits  in  steps  of  various  algorithms 
are  new  (see,  e.g.  Graves  and  Whinston  (1968),  (1970)),  such  usage  is 
certainly  not  widespread.  Hopefully,  improved  confidence  limit  procedures 
will  greatly  increase  their  usage. 

The  author  gratefully  acknowledges  the  support  of  the  Office  of 


Naval  Research. 


APPENDIX  A 


Specific  Input  Instructions  for  the  Computer  Implementation 
of  the  Procedures  based  on  Qlk>  Q2k>  and  Q3fc. 

Card  A.  Title. 

Columns  1-80:  Any  identifying  title. 

Card  B.  Procedures  to  be  used. 

Col.  1:  IQ1K 

IQ1K  *  1  if  the  procedure  based  on  Q  and  the 

X.  K- 

sample  proportions  is  to  be  performed. 

IQ1K  *  0  otherwise. 

Col.  2:  IQ2K 

IQ2K  =1  if  the  procedure  based  on  ^2^  and  t'ie  k 
largest  order  statistics  is  to  be  per¬ 
formed  . 

IQ2K  =  0  otherwise. 

Col.  3:  IQ3K 

IQ3K  =1  if  the  procedure  based  on  Q3fc  and  the 

estimated  derivations  is  to  be  performed. 
IQ3K  =  0  otherwise. 

Card  C.  Controlling  the  amount  of  printed  output. 

Col.  1:  IWRIT1 

IWRIT1  =  1  implies  that  the  sample  proportions,  the 
estimated  alphas  and  distributed  function 
resulting  from  the  procedure  based  on 
will  be  printed. 


IWRITl  ■  0  otherwise. 
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Col.  2:  IWR1T2 

IWRIT2  ■*  1  Implies  that  the  estimated  alphas  and 
distribution  function  resulting  from 
the  procedure  based  on  Q  will  be 
printed . 

IWRIT2  *  0  otherwise. 

Col .  3 :  IWRIT3 

IWRIT3  =  1  implies  that  the  estimated  alphas, 

lambdas,  and  distribution  function  re¬ 
sulting  from  the  procedure  based  on 
will  be  printed. 

IWRIT3  =  0  otherwise. 

Card  D.  Sample  size. 

Col.  1-3:  n  *  the  number  of  independent  observations  on  g. 

(Format  13;  i.e.  a  three  digit  integer  right 
justified.)  1  <_  n  <_  100. 

Card  E.  Observations  in  the  random  sample. 

Col.  1-80:  g^,  i  -  1,  ...,  n. 

g^  *  i-th  independent  observation  on  g. 

(Format  8F10.0;  i.e.,  eight  observations  per  card 
with  the  first  observation  in  columns  1-10  either 
right  justified  or  with  the  decimal  point  included, 
the  second  observation  in  columns  11-20,  etc.) 

Card  F,  Transformation.  (See  Appendix  D  for  a  discussion  of  transforma¬ 
tions.) 

Col.  1-3:  L  (Format  13) 

L  *  0  if  the  g^'s  are  not  to  be  transformed. 
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Card  G. 
Col. 

Card  F. 
Col. 


L  >  0  implies  that  L  non-random  estimates  of  G* 


(namely, 


, ,  to  be  read  in  on  Card  G) 


will  be  used  to  transform  each  independent 

f\j 

observation  to  g^  where 


8i  =  **1  +  *  8(j>  lf  8i  <  8d) 


L-l 


=  WWg  +  Z  WT  g*  +  g* 

j=h+l  J  j-L-h 


L 

Z 


WT. 


lf  8(h)  1  8i  "  8(h+l) 
8i  lf  8i  -  8(L) 


where  g(1)  < 


<  8 

*  * 


(L) 


are  the  ordered 


values  of  g  ,  . . . ,  gL  and  the  WW,  WT.^ . 

WTl  are  the  user  specified  weights  (see  Card  H 
and  possibly  Cards  I  and  J) .  If  L  >  0,  then 

the  procedures  based  on  Q^»  an<*  are 

%  % 

performed  on  the  g^,  instead  of  the 

8r  •**»  8n’ 

0  <  L  <  iO. 

Transformation.  If  L  *  0,  Card  G  is  omitted. 


1-80:  gj ,  j  =  1,  ...,  L.  (Format  8F10.0) 

*  *  *  * 
g^  ■  j-th  non-random  estimate  of  G  ;  gj  <_  G  . 

Transformation.  If  L  =  0,  Card  F  is  omitted. 


1: 


INDEXW 

INDEXW 


0  implies  that  the  weights  WW  and  WT^, 
...,  WT^  are  to  be  read  in. 


Note:  WW  +  Z  WT  ■  1. 

j-1  J 
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INDEXW  =  1  Implies  WW  =  1/(L  +  1)  and 
WT1  =  1/ (L  +  1). 

INDEXW  -  2  implies  WW  =  1/[(L  +  2)  (L  +  l)/2]  and 
WTi  =  (i  +  1)/[(L  +  2)(L  +  l)/2]  . 

Card  H.  Transformation  weight.  If  INDEXW>0,  then  Card  H  is  omitted. 


Col.  1-10: 


J,  the  weight  on  g^.  (Format  F10.0) 


Card  I.  Transformation  weight.  If  INDEXW>0,  then  Card  I  is  omitted. 


Col.  1-80: 


WT,,  j  =  1,  ...,  L.  (Format  8F10.0) 


WTj  is  proportional  to  the  weight  placed  on  the  j-th 

* 

ordered  non-random  estimate  in  constructing  the 


In  the  instructions  below,  read  g^ . g^  in  place  of  g^,  ....  g^ 

if  L  >  0. 

Card  J.  Transformation. 


Col.  1: 


ISUBU 


ISUBU  =1  if  uniform  random  variables  U^,  ..., 

Ur  on  the  range  [0,U  ]  are  to  be  sub¬ 
tracted  from  g, ,  ...»  g  respectively. 
1  n 

The  resulting  observations  will  also 

be  denoted  by  g^ . gn  herein. 

* 

Note  that  G  remains  unchanged. 

ISUBU  =0  if  the  observed  g, ,  . . . ,  g  are  not  to 

l  n  - 

be  transformed  by  subtracting  uniform 


deviates. 


Col.  6-15:  U  .  (Format  F10.0)  This  value  is  ignored  if  ISUBU  *  0. 
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Col.  16-25:  NSEED1 .  (Format  110)  Ignored  If  ISUBU  =  0 
Col.  26-35:  NSEF.D2 .  (Format  110)  Ignored  if  ISUBU  =  0 

NSEEDl,  NSEER2  are  any  2  ten  digit  odd  integers. 
They  are  used  to  initialize  the  composite  uniform 
random  deviate  generator. 

NSEEDl,  NSEED2  <_  2,  147,  4^3,  647. 

Card  K.  Nonnegative  derivatives.  If  the  procedures  based  on  Q  and 
are  neither  one  to  be  used.  Card  K  is  omitted. 

Col.  1-2:  S.  (Format  12) 

S  =  the  number  of  relatively  dense  grid  points, 

gj,  ....  gg,  at  which  the  derivative  of  the  es¬ 
timated  distribution  function  F(*;  a,  G)  is 
explicitly  required  to  be  nonnegative.  For  the 
procedure  based  on  0^ 

g!  -  g(1)  +  (i-i)[«?-g(1)>/s] 

for  i  =  1,  ...,  S.  For  the  procedure  based  on 
Q2k 

8i  =  8(n-k+l)  +  (1“1)^G_8(n-k+l))/S-* 
for  i  =  1 ,  . . . ,  S . 

1  i  s  1  4°. 

• k 

Card  L.  Trial  estimates  of  G  . 

Col.  1-3:  NTRIAL .  (Format  13) 

NTRIAL  =  the  maximum  number  of  trial  estimates, 

* 

GTRIAL,  of  G  to  be  evaluated 

1  <_  NTRIAL  _<  100. 

* 

Card  M.  Trial  estimates  of  G  . 

Col.  1:  INDEXG 

INPEXG  =  0  imp] ies  that  the  NTRIAL  trial  estimates 
* 

of  G  are  being  read  in. 
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* 

INDEXG  =  1  implies  that  the  trial  estimates  of  G 
are  to  be  GTRIAL,  GTR1AL  +  GINC, 

GTRIAL  +  2*GINC,  GTRIAL  +  (NTRIAL-i) 

*GINC. 

* 

Card  N.  Trial  estimates  of  G  .  Only  if  INDEXG  =  0,  is  Card  N  included. 

Col.  1-80:  GTRIAL^,  ....  GHUA^  (Format  8F10.0) 

Note  GTRIAL^  <  ...  <  GTRIALNTRIAL . 

* 

Card  0.  Trial  estimates  of  G  .  Only  if  INDEXG  *  1,  is  Card  0  included. 
Col.  1-10:  GTRIAL.  (Format  F10.0) 

Col.  11-20:  GINC.  (Format  F10.0) 

Card  P.  Stopping  rules. 

Col.  1:  ISTOP 

ISTOP  =  0  implies  that  all  NTRIAL  trial  estimates 
* 

of  G  will  be  considered. 

* 

ISTOP  *  1  implies  that  the  estimation  of  G  will 

* 

stop  as  soon  as  a  trial  estimate  of  G 

is  found  with  a  goodness-of-f it  index 

greater  than  its  critical  value. 

Card  Q.  Using  Q-^*  Include  Card  Q  only  if  IQ3K  =  1. 

Col.  1-8:  XNORM.  (Format  F8.0) 

XN0RM  »  the  100  w  y  -th  percentile  of  the 

standard  normal  distribution  function 

(N(0,1))  when  100  g  y  %  confidence  limits 
* 

on  G  based  on  the  approximate  normality  of 

* 

an/(xl  V-  X-).  -  are  desired.  For  a  95% 

U  G  G 

confidence  limit  XNORM  *  1.645.  (See 
Appendix  G  for  a  table  of  XNORM  values) 
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Card  R.  Using  Q  .  Include  Card  R  only  if  IQ3K  =  1. 

Col.  1-40:  Xjjj+i  y»  m  =  •••»  5.  (Format  5F8.0) 

y  =  the  100  K  y  -th  percentile  of  a  chi-squared 
distribution  with  m  +  1  degrees  of  freedom 

when  100  «  y  %  confidence  limits  on  C  based 
on  are  desired.  (See  Appendix  H  for  a 
table  of  percentiles  of  chi-squared  distri¬ 
butions.) 

Card  S.  Degree  of  F. 

Col .  1 :  IREADM 

IREADM  =  0  implies  that  the  degree,  m,  of  the  ap¬ 
proximating  distribution  function  F  is 
to  be  determined  using  the  procedure 
described  in  Appendix  E. 

IREADM  =  1  implies  that  m  is  to  be  read  in. 

1  ^  i  <  5. 

Col.  2:  ICHKM 

ICHKM  =  0  implies  that  the  degree  of  F  will  not  be 
evaluated  using  the  procedure  described 
in  Appendix  E. 

ICHKM  =  1  implies  that  a  value  for  m  will  be  com¬ 
puted  using  the  procedure  described  in 
Appendix  E.  (If  IREADM  =  0,  then  ICHKM 
is  automatically  set  to  1.)  If  IREADM  =  1, 
the  value  for  m  read  in  is  used  in  the 
procedures  based  on  Q^»  an<*  ^3k’  t*ie 

computed  value  of  m  is  only  for  information 
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and  is  not  used  in  the  procedures  in¬ 
volving  Qlk,  Q2k,  and  Q3k. 

Card  T.  Degree  of  F.  Include  Card  T  only  if  IREADM  =  1. 

Col.  1:  m,  the  degree  of  the  approximating  distribution 

function  F. 

Card  U.  Using  Q^.  Include  Card  U  only  if  IQ1K  =  1. 

Col.  1-2:  k.  (Format  12) 

k  =  the  number  of  sample  proportions  to  be  used  in 
the  confidence  limit  procedure  based  on  Q^- 
1  1  k  £  20. 

2 

Col.  3-12:  Xv  *  (Format  F10.0) 

k,Y 

2 

X.  =  the  100  k  y  -th  percentile  of  a  chi-squared 
k»  Y 

distribution  with  k  degrees  of  freedom 

* 

when  a  100  x  y%  confidence  limit  on  G  is  de¬ 
sired.  (See  Appendix  H  for  a  table  of  per¬ 
centiles  of  chi-squared  distributions.) 

Card  V.  Using  Q  .  Include  Card  V  only  if  IQ1K  =  1. 

Col.  1-80:  G^  ...,  Gk.  (Format  8F10.0) 

G^  *  the  i-th  scale  point  used  in  the  procedure  based 
on  Qlk. 

Card  W.  Using  Q2k.  Include  Card  W  only  if  IQ2K  =  1. 

Col.  1-2:  k.  (Format  12) 

The  k  largest  order  statistics  g^  k+^  ,  ...,  g^ 
will  be  used  to  estimate  the  upper  tail  of  the 
distribution  F  in  the  procedure  based  on  Q2k> 


1  <  k  <  n  <  100. 
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Card  X.  Using  Q2jc*  Include  Card  X  only  if  IQ2k  =  1 

Col.  1-30:  CRITL(l) ,  CRITL(2) .  (Format  2F10.0) 

These  are  the  100  xY  -th  percentiles  of  Q2k  for 

m  =  1,2,  and  3  respectively.  These  percentiles  can 

be  found  in  Tables  F.4,  F.5,  ...  in  Appendix  F.  If 

i  >  3  is  specified,  then  CRITL(2)  is  used  as  an 

approximate  100  *  y  -th  percentile  of  This  level 

100  x  Y  should  correspond  to  the  desired  confidence 

* 

level  in  the  100  x  y  %  confidence  limit  on  G  . 

Card  Y.  Using  Q^.  Include  Card  Y  only  if  IQ3K  =  1. 

Col.  1-2:  k.  (Format  12) 

The  k  largest  order  statistics  g(n  ^+1) . 8(n) 

will  be  used  in  the  procedure  based  on  Q^k  and 
derivatives. 


1  <  k  <  n  <  100. 
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APPENDIX  C 
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APPENDIX  D 

Transformations  of  the  Initial 
Random  Sample  g^ ,  .... 

The  confidence  limit  procedures  described  in  section  2  assume  only 

that 

(i)  g^,  . ..,  are  n  independent  observations  on  a  random 
variable  g, 

•k 

(ii)  the  least  upper  bound  on  g  is  G  ,  and 

(iii)  the  distribution  function,  P(g  ),  of  g  is  well  approximated 
by  a  polynomial  of  degree  at  most  m. 

* 

The  only  characteristic  of  P(g  )  of  real  interest  is  G  .  Hence,  if 

% 

gj  =  transformation  of  g^, 

%  % 

g^,  ....  g^  are  independent, 

*  a. 

G  =  least  upper  bound  on  g,  and 

%  <\j 

V(g  )  =  distribution  function  of  g* 

%  % 

the  sample  g^,  . g  could  also  be  analyzed  by  the  confidence  limit 

*  ^ 

procedures  and  provide  the  desired  information  on  G  .  In  fact,  if  P(g  ) 

is  more  benign  than  P(g  ),  the  analysis  of  g^,  •  .*,  may  provide  a 

* 

better  confidence  limit  on  G  than  that  based  on  g^,  g^* 

At  this  point  in  time  the  most  benign  form  for  P(g  )  is  unknown. 

It  is  suspected  that  a  helpful  form  for  P (g  )  is  roughly  linear  for 
g  <  G  in  a  neighborhood  of  G  with  the  slope  of  $(g  )  near  G  being  neither 
too  near  0  nor  too  large. 

Frequently  when 

* 

G  =  max  g (x)  , 

xeR 
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some  "good"  values  for  x  with  "large"  values  of  g(x)  are  known  or  can 


be  guessed  apriori.  Such  x  values  (say,  x  ,  x  )  and  there 

X  X 

"ft 

associated  g  values  (say  =  g(x^) . g^  =  g(x^))  are  usually 

or  guessed  from  the  physical  characteristics  of  the  problem.  The 


known 


*■»  a 

collection  g^,  ....  g^  represents  L  nonrandom  estimates  of  G  .  Since 

they  are  nonrandom,  the  confidence  limit  procedures  should  not  be  applied 

*  *  *  * 
directly  to  g^,  ....  g^.  However;  the  g^,  g^  can  be  used  to 

"upgrade"  a  random  sample  g^,  ...,  g^.  The  upgraded  random  sample  could 

be  generated  by  the  following  general  transformation 


8i  =  Wgi  +  if  H  "  8(i) 

=  ww8.  +  wTj_h  g(j)  +  g(L)  m.  if  g(h)  <  g±  <g(h+1) 

=  8i  if  gi  i  8(l) 

*  *  *  * 
where  g^  <  ...  <  g^  are  the  ordered  values  of  g^,  ...,  g^  and  WW,  WT^, 

...,  WT  are  nonnegative  weights  with 


1  =  WW  +  T  WT  . 

j-i  J 

o«  * 

Here  g^  is  a  weighted  average  of  and  those  g  s  >  g^  .  The  computer 

implementation  of  the  confidence  limit  procedures  allows  the  user  to 


specify 


WW  =  WTX  =  ...  =  WTl  =  1/ (L+l) 


WW  =  l/[(L+2)(L+l)/2] 

WT±  =  (i+l)/[(L+2)(L+l)/2] 

or  read  in  any  values  for  WW  >0,  WT  ,  ...,  WT .  At  this  time  it  is  not 

X  L 


apparent  what  "optimal"  weights  are. 
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Of  course,  it  is  entirely  feasible  to  create  the  observations  to  be 

analyzed  by  first  transforming  g^,  g^  using  L  nonrandom  estimates 

* 

of  G  and  then  subtracting  random  deviates  . . .  from  these 

1  n 

transformed  variables. 

As  discussed  in  section  3  an  alternative  transformation  of 
8^ «  • • • »  8n  is 

8^  8^  -  u^  ,  i  —  1 ,  . . . ,  n 

where  u^,  . ..,  u^  are  independent  nonnegative  random  variables  with 
greatest  lower  bound  zero.  In  particular  u  being  a  uniform  random 
variable  on  [0,  U  ]  seems  like  a  reasonable  choice  since  this  tends 
to  make  P(g)  linear  near  G*.  Currently,  the  "optimal"  magnitude  of  U* 
relative  to  the  range  of  g  is  unknown. 


i 
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APPENDIX  E 


An  Automated  Procedure  for  Determining  the  Degree 
of  the  Approximating  Polynomial  Distribution  Function 

Sometimes  an  investigation  of  the  degree  of  the  approximating 
polynomial  distribution  function  is  of  interest.  If 

£  ®  £  -i 

F(g;a,G  )  =  l  a  (g  -  G  )J  +  1 
j=l  J 

is  rewritten  as 

*  m 

F(g  ;a  ,G  )  =  l  8  g3  , 

J-0  3 

* 

then  the  smallest  integer  m  ^  2,  say  m  ,  such  that  the  estimate  of 
am  is  not  significantly  different  from  zero  can  be  roughly  determined. 
This  investigation  takes  the  same  form  for  >_he  procedures  based  on 
Q2k  and  but  has  a  different  form  for  the  procedure  based  on  Q  ^ 
These  two  forms  are  discussed  below.  If  the  user  of  the  computer 
implementation  so  requests,  the  value  of  m  for  any  or  all  of  the  con¬ 
fidence  limit  procedures  will  be  determined.  The  user  may  also  specify 

* 

whether  or  not  m  is  to  be  set  equal  to  m  -  1  in  each  of  the  confidence 
limit  procedures. 

For  the  confidence  limit  procedure  based  on  the  determination 

* 

of  m  is  as  follows:  As  described  in  section  2.1,  determine  the  sample 
proportions  p^ . p^  where 

npi  ■  f  of  glt  ....  <  Gt,  1-1,  ....  k. 

Then  the  vector  (p^,  ...,  p^)  has  approximately  a  multivariate  normal 
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/  m 


distribution  with  mean  vector  Y  B.G,^ .  7  &.G.  ^ 

j*0  J  1  j-0  J  k 


and  symmetric  covariance  matrix  =  (v^^)  with 


m 


Thus,  the  weighted  least  squares  estimate  of  B  =  (6,,  . ..,  B  )  is 

1  in 

~  Ta-1  -1  T A-1 

8  =  (X  V  X)  XVj  Y 


which  has  approximately  a  multivariate  normal  distribution  with  mean 

T  -1 

vector  B  and  symmetric  covariance  matrix  X  X 
where 

8=  ( 8^ ,  •••«  » 

X  =  (x^)  with  xij  *  for  i  =  1 . k  and  j  *  0,  m, 

A  A  A 

V1  =  (v^)  with  vltj  =  Pj^d-pjJ/n  for  i  -  1,  ....  k  and 
j  =  1,  ....  k,  and 
T 

x  =  (p^»  • • • *  p^  • 

T~_l  -l  ** 

Furthermore,  if  6  is  0,  then  B  /  (XV.  X)  has  approximately 

in  ml  m,m 

a  t  distribution  with  k-(m  +  1)  degrees  of  freedom.  Since  B  *  ot  , 

m  m 

the  value  of  a  is  assumed  to  be  zero  if 


6„' 


i- 


-  tk-m-l, .95 


where  95  Is  the  95-th  percentile  of  a  t  distribution  with 

k-(m+l)  degrees  of  freedom.  This  is  a  test  of  the  hypothesis  o  *  0 

m 

* 

with  a  significance  level  of  approximately  .10.  The  value  of  m  is 
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the  smallest  Integer  m  >  2  for  which  a  can  be  assumed  to  be  zero  on 

—  m 

the  basis  of  the  above  t  test. 

For  the  confidence  limit  procedures  based  on  either  or 
* 

the  determination  of  m  is  as  follows:  If 


8(n-k+l)  <  *•*  <  8(n) 
are  the  k  largest  order  statistics  from 

m 

ng)  =  I  a.  g2  , 
j=o  J 

then 

z^*F(g(i^)  )  i-n-k  +  1 . n 

are  distributed  like  the  k  largest  order  statistics  from  a  uniform 
distribution.  Analogously  to  section  2.2, 

1 

eh* 

A  I  I 

(  8)  * 

is  minimized  with  respect  to  6.  Letting  m  represent  this  minimum, 

* 

the  value  of  m  -1  is  taken  to  the  smallest  integer  in  such  that 

( 8)  A 
Q_,  is  less  than  the  90-th  percentile  of  Q~,  for  m  =  1  given  in 
z,  tc  ,  m  t-  k 

Appendix  F  (tables  F.4  -  F.9). 

* 

The  computer  implementation  considers  only  m  <_  5.  If  m  seems  to 
* 

be  j>  6,  the  value  of  m  is  set  equal  to  6.  This  corresponds  to  a 
maximum  allowable  value  of  m  in  the  computer  implementation  being  m  =  5. 


(j) 


-  j/n+1 


,(-D 

2ij 


,  (6) 

2k,m 


n 

l 

i=n-k+l 


n 

I 

j  =n-k+l 


m 

i  l 

\h*0 


eh8(i)j'  (1/n+1) 


m 

l 

h-0 


APPENDIX  F 


Percentiles  of  Q„, 
2k 


L 


and 
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Table  F.l  95-th  percentile  of  the  goodness-of-f it  criterion 

based  on  a  Monte  Carlo  simulation  involving  10,000  samples.* 


n 

k  =  5 

10 

15 

20 

25 

30 

35 

40 

45 

50 

10 

13.46 

15 

14.42 

21.85 

20 

14.87 

23.12 

29.38 

25 

15.14 

23.88 

30.72 

36.52 

30 

15.31 

24.37 

31.60 

37.84 

43.41 

35 

15.43 

24.74 

32.22 

38.78 

44.68 

40 

15.52 

25.00 

32.70 

39.47 

45.64 

51.34 

45 

15.60 

25.20 

33.05 

40.01 

46.39 

52.31 

57.85 

50 

15.65 

25.35 

33.34 

40.43 

46.97 

53.06 

58.82 

60 

15.72 

25.61 

33.76 

41.07 

47.82 

54.18 

60.20 

65.99 

70 

15.78 

25.77 

34.07 

41.51 

48.43 

54.97 

61.19 

67.17 

72.92 

80 

15.83 

25.89 

34.29 

41.85 

48.88 

55.55 

61.91 

68.05 

73.96 

79.76 

90 

15.86 

26.00 

34.46 

42.10 

49.22 

56.00 

62.48 

68.73 

74.79 

80.65 

100 

15.88 

26.07 

34.59 

42.31 

49.51 

56.36 

62.91 

69.26 

75.40 

81.42 

information  taken  from  Lurie  (1971)  . 
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Table  F.2  99-th  percentiles  of  the  goodness-of-f it  criterion  Q2k 

based  on  a  Monte  Carlo  simulation  involving  10,000  samples.* 


n 

D 

10 

15 

20 

25 

30 

35 

40 

45 

50 

10 

21.47 

15 

24.21 

31.99 

20 

25.69 

34.89 

41.20 

25 

26.60 

36.72 

43.94 

49.70 

30 

27.23 

37.99 

45.83 

52.26 

57.75 

35 

27.66 

38.90 

47.22 

54.12 

60.12 

40 

28.00 

39.59 

48.27 

55.54 

61.95 

67.88 

45 

28.27 

40.14 

49.10 

56.67 

63.38 

69.44 

75.01 

50 

28.48 

40.59 

49.77 

57.57 

64.53 

70.86 

76.70 

60 

28.81 

41.25 

50.77 

58.95 

66.28 

73.02 

79.26 

85.17 

70 

29.04 

41.73 

51.52 

59.92 

67.54 

74.56 

81.12 

87.32 

93.21 

80 

29.21 

42.09 

52.06 

60.69 

68.49 

75.71 

82.50 

88.94 

95.11 

101.00 

90 

29.35 

42.36 

52.49 

61.27 

69.23 

76.61 

83.60 

90.20 

96.58 

102.69 

100 

29.46 

42.60 

52.84 

61.74 

69.83 

77.36 

84.45 

91.24 

97.71 

103.98 

information  taken  from  Lurie  (1971) 
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Table  F.3  Upper  percentage  points  of  |(  computed  from  (a)  a  Pearson 
Typo  V  approximation,  (b)  The  exact  recurrence  formula  using 
numerical  integration  and  (c)  Monte  Carlo  based  on  10,000  samples  * 


10% 

5% 

i% 

A 

A 

n 

(a)  (6) 

(C) 

(a)  (6) 

(c) 

(a)  («>) 

(e) 

3 

5-75  5-92 

— 

6  98  7-39 

— 

•  9-89  10  19 

— 

4 

7-40  7-50 

— 

8-98  927 

— 

12-78  13-19 

— 

5 

900  908 

— 

1089  11-14 

— 

15-51  1602 

— 

6 

10  56  10-61 

— 

12-74  12-96 

— 

18-09  18-69 

— 

7 

1208  12  13 

— 

14-52  14-71 

— 

20-55  21-11 

— 

8 

1357  13-59 

— 

16-26  16-44 

— 

22-91  2333 

— 

9 

15-03  15-05 

•— 

17-95  18  11 

— 

25-18  25-69 

— 

10 

16-47  16-48 

16  46 

19  61  19  75 

19  83 

27-37  27  79 

27-30 

11 

17-90  17-90 

— 

21  23  2135 

— 

2950  29  88 

— 

12 

1930  19-32 

— 

22-83  •  22  94 

— 

31-56  31  89 

— 

13 

20  70  20-69 

— 

24-41  2448 

— 

33-58’  3386 

— 

14 

22-08  22-09 

— 

25-96  26-05 

— 

35-54  35-73 

— 

15 

2344  2347 

22  96 

27-49  27  56 

26  93 

37-46  37-53 

37-00 

16 

24  80  2483 

— 

29  00  2902 

— 

3935  39  22 

— 

17 

26-14  — 

— 

30  50  — 

— 

4119  — 

— 

18 

27-48  — 

— 

31-98  — 

— 

4301  — 

-- 

19 

28-80  — 

— 

3344  — 

— 

4480  — 

— 

10% 

5% 

1% 

r 

4 

> 

\ 

(<*) 

M 

(a) 

M 

(a) 

ic) 

20 

30  12 

29  77 

34  90 

34  50 

40  56 

45  98 

21 

31  43 

— 

36-34 

— 

48  30 

— 

22 

32  74 

— 

37-77 

— 

50  01 

— 

23 

34  03 

— 

39  19 

— 

51-70 

— 

24 

35-32 

— 

40  60 

— 

5338 

— 

26 

36-61 

— 

42-00 

•  _ 

55  03 

— 

20 

37-89 

— 

43  39 

— 

50  66 

— 

27 

39-16 

— 

44-77 

— 

58  28 

— 

28 

4043 

— 

40-15 

— 

59  89 

— 

29 

41-70 

— 

47-52 

— 

61  48 

— 

30 

42-96 

42  77 

48-88 

48-72 

63  05 

62  64 

3f> 

49  20 

— 

55-59 

— 

70  75 

— 

40 

65-35 

55- 10 

02  10 

61  74 

78  20 

79  55 

45 

61  43 

— 

68  63 

— 

85-44 

— 

50 

67-45 

67-21 

75-00 

7501 

92  53 

93  60 

55 

73  42 

— 

81-30 

_ 

9948 

— 

60 

79  34 

— . 

87-53 

— 

106  31 

— 

65 

85  22 

— 

93  71 

— 

113  05 

— 

70 

91-07 

— 

99  83 

— 

119  70 

— 

75 

96-88 

90-73 

105  90 

105  08 

126  28 

12561 

80 

102-67 

— 

ill-94 

_ 

132  70 

_ 

85 

108-43 

— 

117  04 

— 

139  24 

— 

90 

114-16 

— 

123  91 

— 

145  64 

— 

95 

110  88 

— 

129  84 

— 

152  00 

— 

100 

125-57 

125-97 

135-75 

135  15 

158  31 

167  50 

800 

658  42 

— 

578-11 

— 

018-11 

— 

*  Information  taken  from  Hartley  and  Pfaf fenberger  (1972). 


m 
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Table  F.4  Observed  percentiles  of  for  m  =  1  and  k  *  5  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


Percentile  5  I  10  50  80  90  I  95  I  99 


n 

10  0.7950  1.0949  2.3042  3.3690  3.9015  4.5410  5.3294 

15  0.6751  0.9775  2.1596  3.0425  3.5542  3.9302  4.4962 

20  0.6877  0.9782  2.0696  2.9158  3.3288  3.6932  4.2915 

25  0.6893  0.9596  1.9700  2.7530  3.2200  3.5032  4.0544 

30  0.7273  0.9544  1.9289  2.7361  3.1056  3.3973  3.9597 

35  0.6550  0.9029  1.9346  2.6579  3.1028  3.4212  3.9295 

40  0.7358  0.9684  1.9094  2.6982  3.0836  3.4030  3.8690 

45  0.6800  0.8878  1.8490  2.6046  3.0072  3.2672  3.8270 

50  0.7206  0.9589  1.9374  2.6762  3.0377  3.2993  3.7678 

60  0.6842  0.9035  1.8724  2.5782  2.9308  3.2392  3.6676 

70  0.6255  0.8787  1.8664  2.5900  2.9160  3.1548  3.6470 

80  0.6601  0.9105  1.8391  2.4997  2.8721  3.1398  3.5502 

90  0.7128  0.9529  1.8747  2.6185  2.9622  3.2496  3.5794 

100  0.6180  0.8571  1.8293  2.5376  2.8690  3.1759  3.6190 


Table  F.5  Observed  percentiles  of  Q?^  for  m  =  1  and  k  =  10  based  on  a 
Monte  Carlo  simulation  involving  1500  samples 


Percentile 

5 

10 

50 

80 

90 

95 

99 

n 

10 

3.3711 

4.0552 

7.3316 

10.6786 

12.8215 

15.1200 

20.8051 

15 

2.8167 

3.4484 

5.8781 

8.1584 

9.3515 

10.6328 

13.1150 

20 

2.8702 

3.3154 

5.4976 

7.3223 

8.3734 

9.2977 

11.2154 

25 

2.7925 

3.1467 

5.2457 

6.7980 

7.7112 

8.5010 

9.6934 

30 

2.5955 

3.0817 

5.0570 

6.5695 

7.3799 

7.9939 

9.4950 

35 

2.6555 

3.1218 

4.9751 

6.2796 

7.1655 

7.8611 

8.7823 

40 

2.6070 

3.0222 

4.7909 

6.1524 

6.9258 

7.5151 

8.8534 

45 

2.6782 

3.0145 

4.7416 

6.0532 

6.8252 

7.3820 

8.4659 

50 

2.4862 

2.9308 

4.6528 

5.9321 

6.6772 

7.2681 

8.2132 

60 

2.5016 

2.9023 

| 

4.5525 

5.8266 

6.5429 

7.0334 

7.9444 

70 

2.5207 

!  2.9203 

1 

4.4808 

5.6674 

6.3457 

6.8675 

7.6635 

80 

2.5231 

2.9628 

4.5102 

5.6608 

6.3043 

6.7866 

7.7215 

90 

2.4011 

2.8398 

4.4074 

5.6483 

6.3077 

6.7621 

7.6957 

100 

2.4607 

2.8555 

4.4198 

5.6185 

6.2526 

6.8083 

7.6995 

Percentile 


5 

10 

50 

80 

90 

95 

6.0914 

7.1359 

12.2815 

16.6570 

20.2154 

23.8590 

5.6776 

6.4080 

9.9970 

13.2789 

15.5312 

17.4707 

5.3608 

6.1082 

9.3787 

11.9973 

13.5924 

15.0445 

5.2068 

5.9049 

8.7984 

11.0806 

12.3676 

13.4961 

4.8240 

5.5523 

8.3797 

10.4859 

11.6225 

12.6532 

4.7368 

5.4552 

8.0681 

10.0152 

11.1980 

12.3028 

4.6838 

5.4340 

7.9397 

9.8499 

10.9455 

11.9427 

4.7923 

5.3627 

7.8080 

9.6826 

10.6161 

11.5213 

4.7227 

5.2296 

7.5711 

9.2368 

10.1833 

11.2696 

4.5246 

5.0369 

7.5225 

9.1451 

10.1479 

10.8993 

4.5285 

5.1607 

7.2436 

8.9037 

9.8969 

10.5539 

4.5712 

5.1717 

7.2649 

8.8319 

9.6632 

10.3896 

4.5097 

5.0009 

7.2393 

8.7609 

9.5403 

10.2972 

33.7135 

21.1083 

17.7235 

15.2003 

14.6437 

14.1008 

13.4631 

12.8710 

12.6117 

12.3907 

12.1855 

11.8508 

11.6714 
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Table  F.7  Observed  percentiles  of  Q2fc  for  m  =  1  and  k  =-  20  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


n 
10 
15 

20  9.2579  10.5217  16.5988  22.7815  26.9844  30.6751  38.3473 

25  8.5401  9.5732  14.4469  19.0561  21.8008  24.3088  29.6780 

30  8.0960  8.9815  13.4853  17.0045  19.0373  21.2573  24.8389 

35  7.8583  8.7390  12.5412  15.8006  17.7042  19.5008  22.8097 

40  7.6267  8.5997  12.0263  14.8637  16.5321  17.6523  20.5317 

45  7.4320  8.3000  11.7349  14.3124  15.8265  17.0063  19.8576 

50  7.2564  7.9921  11.3406  13.7053  15.1775  16.5539  18.8688 

60  7.0809  7.8149  10.8401  13.2074  14.5522  15.4355  18.5607 

70  6.9626  7.8092  10.5761  12.7222  13.8563  14.9032  16.8736 

80  6.8612  7.6857  10.3791  12.3866  13.4785  14.3257  16.4240 

90  6.7622  7.5427  10.1685  12.3459  13.3713  14.4311  16.4125 

100  6.9108  7.5716  10.2337  12.1787  13.1416  14.2235  16.1448 
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Table  F.9  Observed  percentiles  of  for  m  =  1  and  k  =  30  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


Percentile 

5 

10 

50 

n 

10 

15 

20 

25 

30 

15.9556 

17.8899 

26.3351 

35 

15.0606 

16.6132 

23.7549 

40 

14.6048 

16.1073 

21.9367 

45 

13.8515 

15.1399 

20.7218 

50 

13.1543 

14.4307 

19.7203 

60 

12.6673 

13.8159 

18.4865 

70 

12.3299 

13.5556 

17.7348 

80 

11.9518 

12.9899 

17.1229 

90 

12.0188 

12.8475 

16.7569 

100 

11.6842 

12.5434 

16.4005 

34.5368 

40.0028 

29.7509 

33.5835 

27.1967 

29.9356 

25.0228 

27.8213 

24.0512 

26.4301 

22.1482 

24.2446 

21.0749 

22.7931 

20.1220 

21.9159 

19.7140 

21.3977 

19.1245 

20.5820 

45 .4640 
37.1985 
32.6829 
30.6415 
28.8160 
26.3576 
24.4371 
23.5572 
22.7690 
22.0335 


59.5800 

44.5941 

39.0256 

36.2437 

33.0465 

29.1589 

28.0009 

26.7444 

25.3704 

25.0725 
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Table  F.10  Observed  percentiles  of  for  m  =  2  and  k  =  5  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


Percentile 

5 

10 

50 

80 

90 

95 

99 

n 

10 

0.4613 

0.6782 

1.7502 

2.7411 

3.3214 

3.8755 

4.6372 

15 

0.4501 

0.6765 

1.6995 

2.4604 

2.9661 

3.3802 

4.0154 

20 

0.4174 

0.6645 

1.6416 

i 

2.4443 

1 

2.8460 

3.1589 

3.7249 

25 

0.4632 

0.6502 

1.6027 

2.3419 

2.7016 

3.0544 

3.7034 

30 

0.4344 

0.6789 

1.6016 

2.2395 

2.5904 

2.8859 

3.4380 

35 

0.3967 

0.6054 

1.5620 

2.2351 

2.6275 

2.9467 

3.5036 

40 

0.4358 

0.6133 

1.5123 

2.2375 

2.5901 

2.8693 

3.5123 

45 

0.4004 

0.6090 

1.5335 

2.1455 

2.5141 

2.8219 

3.2461 

50 

0.3893 

0.6231 

1.5539 

2.2010 

2.5994 

2.8621 

3.3066 

60 

0.4146 

0.6190 

1.5171 

2.1555 

2.5063 

2.7982 

3.3952 

70 

0.4869 

0.6595 

;  1.5227 

I 

2.1448 

2.5051 

2.7531 

3.1990 

80 

0.3795 

0.6008 

1.4664 

2.0944 

2.4239 

2.7459 

3.2971 

90 

0.4492 

0.6889 

1.4855 

2.0973 

2.4535 

2.7124 

3.3451 

100 

0.4193 

0.6387 

i 

1.4665 

2.1283 

2.4971 

2.7880 

3.1881 
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Table  F.ll  Observed  percentiles  of  Q2fc  for  m  =  2  and  k  =  10  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


Percentile 

5 

10 

i 

50 

80 

90 

95 

99 

- 

n 

10 

2.7735 

| 

3.4664 

6.3194 

9.0145 

1 

11.0287 

12.5317 

16.6531 

15 

2.4236 

1 

2.9753 

5.1627 

7.1074 

8.4276 

9.4173 

11.2169 

20 

1  2.4617 

2.8621 

4.8016 

6.4501 

7.3404 

8.0265 

9.4346 

25 

2.2138 

2.6814 

4.5273 

5.9655 

6.7507 

7.5323 

8.8709 

30 

2.2791 

2.6962 

4.4365 

5.7928 

6.5952 

7.2239 

8.7974 

35 

2.2573 

2.6721 

4.2996 

5.6440 

6.3629 

6.9724 

7.9919 

40 

2.2259 

2.6656 

4.2738 

5.5084 

6.2222 

6.7973 

7.9441 

45 

2.1947 

2.5782 

4.2543 

5.4622 

6.1280 

6.8178 

7.8115 

50 

2.1860 

2.5545 

4.1348 

5.3299 

6.0032 

6.5015 

7.6552 

60 

2.1459 

2.5384 

4.0685 

5.2358 

5.7932 

6.3799 

7.2054 

70 

2.0723 

2.4636 

4.0438 

5.1125 

5.7523 

6.2126 

6.9384 

80 

2.0873 

2.4621 

3.9945 

5.1090 

5.7458 

6.2483 

7.2358 

90 

2.1828 

2.5129 

3.9601 

5.0589 

5.6437 

6.1858 

7.0296 

100 

2.1810 

2.5089 

3.9695 

5.0775 

i 

5.5919 

6.0206 

6.7823 

-79- 


Table  F.12  Observed  percentiles  of  for  m  =  2  and  k  =  15  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


Percentile 

5 

10 

50 

80 

90 

95 

99 

n 

10 

: 

1 

15 

5.4998 

6.4797 

10.5848 

14.2243  I 

| 

16.9938 

20.2635 

26.6316 

20 

5.1906 

5.8855 

9.2666 

12.1887 

14.1269 

15.7781 

19.5845 

25 

4.7518 

5.4342 

8.3780 

10.7441 

12.1953 

13.5227 

15.9983 

30 

4.3919 

5.0953 

7.8047 

9.9288 

11.1551 

12.2020 

14.3209 

35 

4.4576 

4.9903 

7.5455 

9.4857 

10.5866 

11.4383 

13.8742 

40 

4.3089 

5.0156 

7.3988 

9.2471 

10.4213 

11.2213 

12.9824 

45 

4.2903 

4.8581 

■  7.2590 

8.9729 

10.0068 

10.9088 

12.7422 

50 

4.4317 

4.9606 

7.1508 

8.9278 

9.9109 

10.7047 

12.5022 

60 

4 . 1014 

4.6746 

6.9142 

8.4749 

9.4833 

10.3124 

11.7445 

70 

4.1753 

4.6893 

6.7713 

1 

8.4153 

9.2361 

10.0144 

11.4842 

80 

4.1534 

4.6429 

6.7791 

8.3617 

9.1988 

9.7937 

11.1319 

90 

4.1527 

4.6438 

6.7590 

8.1571 

8.9440 

9.6718 

10.7496 

100 

4.0700 

4.6384 

6.5378 

8.0925 

8.8913 

9.6060 

10.8851 

-80- 


Table  F.13  Observed  percentiles  of  Q2fc  for  m  =  2  and  k  =  20  based  on  a 
Monte  Carlo  simulation  involving  1500  samples. 


Percentile 

5 

10 

50 

80 

90 

— 

95 

99 

n 

10 

15 

20 

8.5894 

9.7797 

14.9154 

20.1513 

23.5669 

25.8466 

33.6911 

25 

7.9831 

8.9248 

13.2781 

16.9533 

19.1330 

21.4918 

26.6315 

30 

7.5345 

8.4257 

12.1439 

15.3297 

17.1796 

18.8577 

22.9272 

35 

7.3227 

8.2013 

11.7392 

14.5209 

16.1844 

17.8072 

21.3554 

40 

7.0572 

7.8364 

11.1554 

13.6066 

15.0583 

16.4068 

19.1958 

45 

6.6492 

7.5481 

10.7026 

13.1403 

14.4691 

15.5158 

18.1505 

50 

6.7204 

7.4707 

10.5959 

12.7150 

14.0975 

15.1015 

17.0252 

60 

6.6992 

7.2944 

10.1405 

12.2380 

13.5588 

14.3838 

16.4226 

70 

6.4482 

7.1605 

10.0402 

12.0705 

13.1646 

14.0537 

16.1698 

80 

6.3564 

7.0138 

9.6327 

11.5153 

12.5613 

13.4435 

15.2592 

90 

6.2466 

6.8562 

9.5335 

11.4035 

12.4791 

13.1713 

14.8876 

100 

6.4103 

6.9463 

9.3183 

11.3160 

12.3835 

13.3123 

14.4369 

Table  F.14  Observed  percentiles  of  for  m  =  2  <‘and  k  =  25  based  on 
a  Monte  Carlo  simulation  involving  1500  samples. 


90 

95 

99 

29.0540 

33.0684 

41.8129 

24.6344 

27.5356 

33.3442 

22.4879 

24.8890 

30.7655 

20.9758 

23.0225 

25.9440 

20.0668 

21.5444 

25.6196 

19.3772 

20.7472 

23.5817 

17.9238 

19.2319 

21.5942 

16.9985 

18.1400 

20.2148 

16.4483 

17.5788 

20.3181  ! 

16.2174 

17.1329 

18.9588 

15.9483 

17.0049 

18.8317 

-82- 


Table  F.15  Observed  percentiles  of  for  m  =  2  and  k  =  30  based  on 

a  Monte  Carlo  simulation  Involving  1500  samples. 


Percentile 

|  5 

1 

1  10 

50 

80 

90 

95 

99 

n 

10 

1 

i 

15 

20 

25 

30 

15.0876 

16.9618 

24.5298 

30.7084 

34.6084 

39.1290 

48.3428 

35 

14.4291 

15.9360 

22.1135 

27.8407 

31.6929 

35.4093 

42.3229 

40 

13.3173 

14.6457 

20.0889 

24.4914 

27.5545 

30.1500 

35.3132 

45 

13.3775 

14.4755 

19.5084 

23.7013 

26.4497 

28.5206 

32.8791 

50 

12.2556 

13.3886 

18.7108 

22.5190 

24.8608 

26.6890 

30.5328 

60 

12.1600 

13.1408 

17.6514 

20.8903 

23.0409 

24.8029 

27.5976 

70 

11.9319 

12.9337 

16.8397 

19.8259 

21.5450  j 

22.9388 

25.5781 

80 

11.2707 

12.3369 

16.2566 

i 

18.9258 

20.4412 

21.8323 

25.2829 

90 

11.1630 

12.0334 

15.7477 

18.7200 

20.5235 

21.9124 

24.9271 

100 

11.2597 

12.1157 

15.6233  ' 

18.2995 

19.8963 

21.1062 

23.8996 

-83- 


APPENDIX  G 
XNORM  Values 


idence  level 

XNORM  value 

99.9% 

3.09 

99.5% 

2.58 

99.0% 

2.33 

98.0% 

2.05 

97  .0% 

1.88 

96.0% 

1.75 

95.0% 

1.64 

90.0% 

1.28 

85.0% 

1.04 

80.0% 

.84 

70.0% 

.52 

60.0% 

.25 

50.0% 


00 


-84- 

APPENDIX  H 


Percentiles  of  Chi-Squared  Distributions 


*D.I 

’  X?005 

2 

x.oi 

2  2 
X.025  X .05 

2 

x.io 

2 

X*20 

2 

X  .30 

2 

x.50 

2 

x.70 

2 

X.80 

2 

X.90 

2 

X  •  95 

2 

X.975 

2 

X.99 

X-995 

1 

.000 

.001 

.004 

.016 

.064 

.148 

.455 

1.07 

1.64 

2.71 

3.84 

5.02 

6.63 

7.88 

2 

.010 

.020 

.051 

.103 

.211 

.446 

.713 

1.39 

2.41 

3.22 

4.61 

5.99 

7.38 

9.21 

10.6 

3 

.072 

.115 

.216 

.352 

.584 

1.00 

1.42 

2.37 

3.66 

4.64 

6.25 

7.81 

9.35 

11.3 

12.8 

4 

.207 

.297 

.484 

.711 

1.06 

1.65 

2.20 

3.36 

4.88 

5.99 

7.78 

9.49 

11.1 

13.3 

14.9 

5 

.412 

.554 

.831 

1.15 

1.61 

2.34 

3.00 

4.35 

6.06 

7.29 

9.24 

11.1 

12.8 

15.1 

16.7 

6 

.676 

.872 

1.24 

1.64 

2.20 

3.07 

3.83 

5.35 

7.23 

8.56 

10.6 

12.6 

14.4 

16.8 

18.5 

7 

.989 

1.24 

1.69 

2.17 

2.83 

3.82 

4.67 

6.35 

8.38 

9.80 

12.0 

14.1 

16.0 

18.5 

20.3 

8 

1.34 

1.65 

2.18 

2.73 

3.49 

4.59 

5.53 

7.34 

9.52 

11.0 

13.4 

15.5 

17.5 

20.1 

22.0 

9 

1.73 

2.09 

2.70 

3.33 

4.17 

5.38 

6.39 

8.34 

10.7 

12.2 

14.7 

16.9 

19.0 

21.7 

23.6 

10 

2.16 

2.56 

3.25 

3.94 

4.87 

6.18 

7.27 

9.34 

11.8 

13.4 

16.0 

18.3 

20.5 

23.2 

25.2 

11 

2.60 

3.05 

3.82 

4.57 

5.58 

6.99 

8.15 

10.3 

12.9 

14.6 

17.3 

19.7 

21.9 

24.7 

26.8 

12 

3.07 

3.57 

4.40 

5.23 

6.30 

7.81 

9.03 

11.3 

14.0 

15.8 

18.5 

21.0 

23.3 

26.2 

28.3 

13 

3.57 

4.11 

5.01 

5.89 

7.04 

8.63 

9.93 

12.3 

15.1 

17.0 

19.8 

22.4 

24.7 

27.7 

29.8 

14 

4.07 

4.66 

5.63 

6.57 

7.79 

9.47 

10.8 

13.3 

16.2 

18.2 

21.1 

23.7 

26.1 

29.1 

31.3 

15 

4.60 

5.23 

6.26 

7.26 

8.55 

10.3 

11.7 

14.3 

17.3 

19.3 

22.3 

25.0 

27.5 

30.6 

32.8 

16  1 

5.14 

5.81 

6.91 

7.96 

9.31 

11.2 

12.6 

15.3 

18.4 

20.5 

23.5 

26.3 

28.8 

32.0 

34.3 

17 

5.70 

6.41 

7.56 

8.67 

10.1 

12.0 

13.5 

16.3 

19.  i 

21.6 

24.8 

27.6 

30.2 

33.4 

35.7 

18 

6.26 

7.01 

8.23 

9.39 

10.9 

12.9 

14.4 

17.3 

20.6 

22.8 

26.0 

28.9 

31.5 

34.8 

37.2 

19 

6.83 

7.63 

8.91 

10.1 

11.7 

13.7 

15.4 

18.3 

21.7 

23.9 

27.2 

30.1 

32.9 

36.2 

38.6 

20 

7.43 

8.26 

9.59 

10.9 

12.4 

14.6 

16.3 

19.3 

22.8 

25.0 

28.4 

31.4 

34.2 

37.6 

40.0 

21 

8.03 

8.90 

10.3 

11.6 

13.2 

15.4 

17.2 

20.3 

23.9 

26.2 

29.6 

32.7 

35.5 

38.9 

41.4 

22 

8.64 

9.54 

11.0 

12.3 

14.0 

16.3 

18.1 

21.3 

24.9 

27.3 

30.8 

33.9 

36.8 

40.3 

42.8 

23 

9.26 

10.2 

11.7 

13.1 

14.8 

17.2 

19.0 

22.3 

26.0 

28.4 

32.0 

35.2 

38.1 

41.6 

44.2 

24 

9.89 

10.9 

12.4 

13.8 

15.7 

18.1 

19.9 

23.3 

27.1 

29.6 

33.2 

36.4 

39.4 

43.0 

45.6 

25 

10.5 

11.5 

13.1 

14.6 

16.5 

18.9 

20.9 

24.3 

28.2 

30.7 

34.4 

37.7 

40.6 

44.3 

46.9 

26 

11.2 

12.2 

13.8 

15.4 

17.3 

19.8 

21.8 

25.3 

29.2 

31.8 

35.6 

38.9 

41.9 

45.6 

48.3 

27 

11.8 

12.9 

14.6 

16.2 

18.1 

20.7 

22.7 

26.3 

30.3 

32.9 

36.7 

40.1 

43.2 

47.0 

49.6 

28 

12.5 

13.6 

15.3 

16.9 

18.9 

21.6 

23.6 

27.3 

31.4 

34.0 

37.9 

41.3 

44.5 

48.3 

51.0 

29 

13.1 

14.3 

16.0 

17.7 

19.8 

22.5 

24.6 

28.3 

32.5 

35.1 

39.1 

42.6 

45.7 

49.6 

52.3 

30 

13.8 

15.0 

16.8 

18.5 

20.6 

23.4 

25.5 

29.3 

33.5 

36.2 

40.3 

43.8 

47.0 

50.9 

53.7 

40 

20.7 

22.1 

24.4 

26.5 

29.0 

32.3 

34.9 

39.3 

44.2 

47.3 

51.8 

55.8 

59.3 

63.7 

66.8 

50 

28.0 

29.7 

32.3 

34.8 

37.7 

41.4 

44.3 

49.3 

54.7 

58.2 

63.2 

67.5 

71.4 

76.2 

79.5 

60 

35.5 

37.5 

40.5 

43.2 

46.5 

50.6 

53.8 

59.3 

65.2 

69.0 

74.4 

79.1 

83.3 

88.4 

92.0 

*D.F.  s  Degrees  of  Freedom 
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